Packet sharing data transmission system and relay to lower latency

ABSTRACT

The system is for combining and transmitting data fragments for multiple customer ports sent over a first packet-switched computer network to a trunk network and redistributed over a second packet-switched computer network, each customer port having a preselected bandwidth designation. The system provides a First-In First-Out (FIFO) register to capture incoming data bits from each customer port in parallel and speeds equal to or greater than 1 Gbps, providing selectors connected to the FIFO outputs. Each selector is capable of mapping individual incoming data bits from different customer ports and combining them into the same trunk fragments. Each trunk fragment receives data bits from customer ports. by selector values for each customer. The filling sequence for each fragment is repeated until sufficient trunk fragments form a filled trunk frame.

This application claims priority from U.S. Provisional Application No.61/877,385 filed Sep. 13, 2013.

This application pertains to systems and methods for management,construction and control of data packets in order to lower latency oftransmissions via and including radio frequency (RF), free space optics(FSO), (collectively “wireless”) and/or fiber optics transmission media.

BACKGROUND

Ethernet and packet-switched Internet Protocol (IP) networks are systemsfor transmitting data between different points. These systems may beknown as “point-to-point” or “contention-based” systems. In manycircumstances all transmitters contend for network resources and alltransmitters may transmit simultaneously. Due such transmissionprocesses network resources may be oversubscribed, bottlenecks may occuror data may be delayed or lost, resulting in network impairment and highlatency rates.

Conventional networks comprise a plurality of Local Area Network (LAN)endpoints, for example computers connected to an Ethernet LAN. Endpointsmay be coupled to one or more LAN switches that connect to one or moreadditional LAN endpoints. If too many packets are simultaneouslytransmitted by other endpoints, the LAN switch may have a queueoverflow, causing latency and/or packets to be lost. (“Packets” meandatagrams in a LAN or Wide Area Network (WAN) environment. In a LANenvironment, packets are sometimes called “frames.” In a packet-switchedWAN environment, packet-switching devices are normally referred to as“routers”).

Delivery protocols for resolving congestion and replacing droppedpackets are well known, such as Transmission Control Protocol (TCP). Butsuch solutions may be inappropriate for point-to-point environments andRF transmissions where latency below one microsecond is required, theround-trip propagation delay between stations is excessive and even thelatency of waiting for another's single packet to be sent is excessive.Scheduling the transmission of packets by partitioning the packets intodiscrete frames and subframes is known in order to combat latencyissues, but is insufficient in low latency systems. It is possible toschedule packets for delivery during time slots in the subframescorresponding to empirically determined conditions, but there can be adelay of a one to many subframes before data starts to flow for a givenapplication. Based on the packet size and underlying network bandwidth,some varying fraction of each time slot would be actually used totransmit a packet. Assuming a packet size of 125 bytes (1,000 bits) anda port allocated 10 mbps, a single 100-microsecond time slot would beused to transmit each 125 byte packet. With a packet size of 1,500bytes, twelve of the 100 microsecond intervals would be consumed by eachpacket transmission.

Configuration maps are known to provide a data structure indicating timeslots that have already been allocated to other transmitters forreception by the receiving node (or, alternatively, time slots that havenot yet been allocated, or, alternatively, time slots that arecandidates for transmission). A reception map is a data structure thatindicates time slots during which transmission to the intended receivingnode would not conflict with other transmitters. Although there are manyways of representing such a map, one approach is to use a bitmap whereineach bit corresponds to one time slot, and a “1” indicates that the timeslot has been allocated to a transmitting node, and a “0” indicates thatthe time slot has not yet been allocated. If a 1,232-nanosecond deliveryinterval were divided into 1 nanosecond time slots, there would be 1,232bits in the reception map. What makes this approach unique is thatinstead of dividing port traffic on the basis of packets which can be12,000 bits long or larger, this approach divides up and mixes trafficfrom different ports at the bit level, assuring port data moves throughthe system limited only by assigned bandwidth and not effected by thetraffic on other ports.

Such processes for handling packets and the assignment of a packethandling identifier are known in order to share resources betweenretransmitted packets and other transceiver functions. A packet can beforwarded directly to another communication device (or layer) or it mayhave a specific packet handling identifier, such as a Quality of Service(QOS) level. The QOS level of a packet indicates the importance ofcertain service metrics of one or more packets. A QOS metric is delay(or latency) and Packet Error Rate (PER) and include Bit Error Rate(BER), data rate, delay variation (or jitter), packet loss rate or timebetween error events (TBE). Voice or data (e.g., trading) traffic mayhave very low latency requirements and low packet error rate.

Telecommunication networks use a wide variety of infrastructure methodsfor transmitting data depending on a variety of factors including thebandwidth of the connection, the need of security for the data, the typeof data being transmitted, and the routing of the signals between thesource and the destination. Many conventional methods compress thetransmitted data into packets that include header information that isused by the network during the transmission process. The use of framesat the data link layer to transport a data payload from a packet may bereferred to as “encapsulating” the data. Such conventional encapsulationmethods include High-Level Data Link Control (HDLC), Point-to-PointProtocol (PPP) in HDLC-like framing (Internet Engineering Task Force RFC1662), Multiprotocol Encapsulation (MPE), Generic Stream Encapsulation(GSE), and Unidirectional Lightweight Encapsulation (ULE).

Well known transmission systems include the Ethernet protocol ISO/IEC8802-3 (ANSI/IEEE Std. 802,3, 1993 edition) that defines a half-duplexmedia access mechanism and permits all stations to access networkchannel with equality. Each station includes an Ethernet interface cardthat in some cases may use carrier-sense multiple-access with collisiondetection (CSMA/CD) to listen for traffic on the media. A station havingdata to send will attempt to access the channel by waiting apredetermined time after the deassertion of a receive carrier on themedia, known as the interpacket gap (IPG) interval. In some cases a fullduplex environment may be provided and referred to as IEEE 802.3x, FullDuplex with Flow Control-Working Draft (0.3). Traffic is notdistinguished or prioritized over the medium. The environment provides atwo-way, point-to-point communication link between two network stations.In some cases two stations can simultaneously transmit and receiveEthernet data packets between each other without collision. An exampleof data evaluation may include setting the first value of the framedelimiter to 0xC7 (11000111) and setting the second value of the framedelimiter to 0x47 (01000111).

The IEEE 802.3z Task Force has defined shared and full-duplex gigabitEthernet. Slot time of 512 bytes (4,096 bits) is available withoutincreasing the minimum frame length, and providing frame bursting inwhich a station sends several frames separated by the extend carriersymbols in a single burst. Gigabit Ethernet (full duplex) operates by astation operating according to the conventional CSMA/CD protocol whenattempting to transmit the first packet (at speeds under 10G).

IEEE 802.12-1995, “Demand Priority Access Method, Physical Layer andRepeater Specification for 100 Mb/s Operation,” also known as the VGANYLAN network, uses a centralized hub to arbitrate among the requestsfrom network stations. The hub grants access to the stations in a roundrobin fashion and the VG ANYLAN network requires control by a centralhub.

Prior art systems such as Infiniband that use switched fabric topologyor OSPF-TE require five seconds or more to recognize one link isunusable and when the RE link is up but has a high Bit Error Rate (BER)there may be no failover at all. Once failover is achieved, all of thedropped packets are lost in a UDP/IP low-latency network. Once the RFsystem returns to normal it may take considerable time for the system tostart using RF link again.

However, none of the above systems sufficiently resolve latency issueswhere gigabit or higher transmissions of data, such as electronictrading, which must have minimal latency and few or no missed packetsand no waiting for and no impact from other traffic which shares thebackbone latency. Also, each of the above systems fail to provide datatransmission with consistently low latency. A system that improves uponthe above technologies and protocols that provides data transmissionwith latency rates below the single, minimum-sized-packet serializationtime is needed for point to point networks.

For example, a system for sending 1232 data bits per 1248 total bits(equivalent to 154/156 encoding) with no further encoding or loss ofbandwidth, is distinguished from traditional encoding methods whichprovide 8 data bits per 10 total bits (8/10 encoding), 64 data bits per66 total bits (64/66 encoding) and other encoding schemes which decreasethe available data bandwidth as a percentage of the total bandwidth.

Also a system having a step of sending trunk fragments that arescrambled by 16 bit Exclusive-Or (XOR) Look Up Table (LUT) comprised of16 to 1,232 scrambling data bits to create a scrambled trunk fragment(“XOR LUT”), but not scrambled by the self-organizing scramblingpolynomial described by IEEE 802.3-2008 section 49.2.6 (EthernetScrambler), however, the Ethernet Scrambler has the side-effect ofcreating three bit errors for every bit which is flipped by weather orother transmission errors. The XOR LUT described herein does not createextra bit errors.

SUMMARY

The present invention provides for a method of combining andtransmitting data fragments for multiple customer ports C1 to Cn sentover a first packet-switched computer network to a trunk network andredistributed over a second packet-switched computer network, eachcustomer ports having a preselected bandwidth designation, the methodcomprising the steps of providing a First-In First-Out (FIFO) registerto capture incoming data bits from each customer ports in parallel andspeeds equal to or greater than 1 Gbps. The invention provides selectorsconnected to the FIFO outputs, each selector capable of mappingindividual incoming data bits from different customer ports andcombining them into the same trunk fragments where the number of bits ineach trunk frame is a pro rata share of the bandwidth designated to eachcustomer ports C1 to Cn. The invention includes filling each trunkfragment with data bits from customer ports C1 to Cn, by selector valuesfor each customer, repeating the filling sequence for each fragmentuntil sufficient trunk fragments form a filled trunk frame, sending eachfilled trunk frame across the trunk network having a transmission ratethat is less than the sum of all customer incoming bandwidths but equalto the sum of the trunk bandwidth allocation for all customers andmoving the filled trunk frames from the first incoming packet-switchedcomputer network, through the bandwidth-limited trunk to the secondpacket-switched computer network and on to an intended receiving networkendpoint.

In an embodiment, the invention has at least 2 customers and up to 64customers connected to the first packet switched computer network. In anembodiment, the invention further comprising the step of filling of16-bit trunk fragments using 4-bit selectors to select each customerdata bit. In an embodiment, the invention further comprising the step ofproviding selectors that receive data bits from every customer on thefirst incoming packet-switched computer network as soon as the firstdata bits arrive without waiting for a complete data packet from anycustomer. In an embodiment, the invention further comprising the step ofscrambling the 16 bit trunk fragment using a 16 bit Exclusive-Or (XOR)Look Up Table (LUT) comprised of 16 to 1232 scrambling data bits tocreate a scrambled trunk fragment. In an embodiment, the inventionfurther comprising the step of adding a trunk frame cyclic redundancycheck (CRC) of 12-bits to the filled trunk frame. In an embodiment, theinvention further comprising the step of adding a trunk frame 4-bitcount to the filled trunk frame. In an embodiment, the invention furthercomprising the step of resetting the selectors, using the XOR LUT andbeginning the next trunk frame with no break or idle section betweenfilled trunk frames.

In an embodiment, the invention further comprising the step ofencrypting customer data by combining all customer bits using the XORscrambler bits, the pro rata share for all customers, the method used tospread customer data within trunk frames and the map which divides upcustomer bits by timeslice. In an embodiment, the invention furthercomprising the step of sending trunk fragments that are scrambled by XORLUT, but not scrambled by a self-organizing scrambling polynomial. In anembodiment, the invention further comprising the step of providing areceiver with an XOR LUT to descramble and break up the trunk fragmentsinto data bits for each customer.

In an embodiment, the invention further comprising the step of receivingthe trunk fragments at the receiving network end point of the secondpacket-switched computer network. In an embodiment, the inventionfurther comprising the step of detecting missing data fragments and databit errors in the incoming trunk fragments. In an embodiment, theinvention further comprising the step of sending 1,232 data bits per1,248 total bits (equivalent to 154/156 encoding) with no furtherencoding or loss of bandwidth. In an embodiment, the invention furthercomprising the step of correcting data bit errors and resynchronizingwhen the trunk fragments are lost. In an embodiment, the inventionfurther comprising the step of providing each customer with a FIFOregister, each customer assigned FIFO register for holding partiallyreceived customer data packets. In an embodiment, the invention furthercomprising the step of providing selectors that match customer data bitswith the corresponding customer assigned FIFO register.

In an embodiment, the invention further comprising the step of receivingdata fragments by the customer assigned FIFO register and locating anend-of-frame block type field. In an embodiment, the invention theend-of-frame being is designated as idle block type field. In anembodiment, the invention further comprising the step of transmittingthe filled trunk frame as soon as enough data arrives to allow thecontinuous sending of all customer packets. In an embodiment, theinvention having each customer's pro rata share of bandwidth equal thepercentage of total subscribed bandwidth divided by 1,232 bits. In anembodiment, the invention of the selector uses 3, 4, 5 or 6 bit indexesto assemble 8 bit, 16 bit, 32 bit or 64 bit values to fill each trunkframe. In an embodiment, the invention having the completed packetbetween 1,232 bits and 1,520 bytes. In an embodiment, the inventionincluding having data bits in every trunk frame. In an embodiment, theinvention providing the customer ports bits that are spread out withinevery trunk frame so as to prevent waiting when multiple customers senda number of packets all at the same time. In an embodiment, theinvention each customer ports's data bits are fragmented among multipletrunk fragments and frames.

In an embodiment, the trunk backbone has throughput of at least 199 Kb.

In an embodiment, the invention having data packets that are sent usingconfigurable logic blocks (CLB). In an embodiment, the invention havingthe CLB as a layer 1+ field-programmable gate array or applicationspecific integrated circuit (ASIC), the first packet switched computernetwork providing a relay for point to point wireless, Infiniband orEthernet transmissions. In an embodiment, the invention having latencybelow 1 microsecond round-trip, assuming a trunk transmission speed ofat least 10 Mbps.

In an embodiment, the invention comprising the step of, at an intendedreceiving network endpoint, generating a reception map on the basis ofpreviously allocated time slots from other transmitting networkendpoints and the reception map comprises a bitmap, wherein each databit corresponds to one of a plurality of timeslots, each data bitindicating whether that corresponding timeslot has previously beenallocated. In an embodiment, the invention comprising the step ofperiodically synchronizing, as between the transmitting network endpointand the receiving network endpoint, a time period on which the proposedtransmission map is used and without synchronization among networkresources. In an embodiment, the invention synchronizing step comprisesthe step of using a connection over which a synchronization signal istransmitted, separate and apart from any network connection and thesynchronizing step comprises the step of transmitting synchronizationpackets over the network.

Another aspect of the invention includes a method of transmittingInternet Protocol (IP), Infiniband or Ethernet packets for multiplecustomer ports C₁ to C_(n) over a packet-switched computer network,comprising the steps of providing a First-In First-Out (FIFO) registerhaving sixteen, four bit selectors, each selector capable of mapping foronly one customer ports, creating a 16 bit trunk fragment, filling eachtrunk fragment from customer ports C₁ to C_(n) by 4 bit selector valuesfor each customer ports, repeating the filling sequence for each 16 bitfragment until sufficient trunk fragments form a filled trunk framehaving at least 1,232 customer ports bits plus 16 bits of CRC and framecount and transmitting each filled trunk frame across a trunk network.

In an embodiment, the invention comprising the step of using only theidle frames sent before and after customer ports packets to distinguishbetween customer ports data and idle frames in order to provide layer 1+transmission, as all possible packets of all formats may be sent and aresurrounded by idle frames. In an embodiment, the invention having thefilled trunk fragment of 16 bits or smaller. The method of claim 40wherein each customer ports C₁ to C_(n) has bits contained in everyfilled trunk frame. In an embodiment, the invention comprising the stepof forwarding any and all customer ports packets to the correctdestination based on the relay port from which it was received. In anembodiment, the invention comprising the step of forwarding all networkadministration, routing, discovery, query, reply, broadcast, unicast andmulticast packets to the destination port on the second packet-switchingnetwork without changing the configuration or routing of the relaydevice and without sending those packets anywhere except the destinationport for the sending customer ports. In an embodiment, the inventionincluding each trunk frame being fragmented. In an embodiment, theinvention having a trunk backbone throughput that is at least 199 Kb.

In another aspect, the invention provides a low latency relay comprisinga First-In First-Out (FIFO) register to capture incoming data packetsfrom each customer ports in parallel and speeds equal to or greater than1 Gbps selectors connected to the FIFO outputs, each selector capable ofmapping individual incoming bits from different customers and combiningthem into the same output trunk fragments in order to create trunkfragments where the number of bits indexed in each trunk frame for eachcustomer ports is a pro rata share of bandwidth designated to eachcustomer ports C₁ to C_(n), the selectors filling each trunk fragmentwith bits from customers C₁ to C_(n), by selector values for eachcustomer ports, selectors repeating the filling sequence for eachfragment until sufficient trunk fragments form a filled trunk frame, anExclusive-Or (XOR) Look Up Table (LUT) comprised of 16 to 1232scrambling bits to create scrambled trunk fragments and a trunk networkfor transmitting each filled and scrambled trunk frame at a transmissionrate that is less than a combination of all customer ports incomingbandwidths from a first incoming packet-switched computer network.

In an embodiment, the invention provides between 2 and 64 customers perrelay and 16 to 64, 4 bit selectors to fill each trunk fragment. In anembodiment, the invention comprising a first packet-switched computernetwork linked to a trunk network, linked to a second packet-switchedcomputer network and the filled and scrambled trunk frame received bythe second packet-switched computer network and being unscrambled by anXOR LUT.

In an embodiment, the invention having selectors that receive data bitsfrom every customer ports as soon as the first bits arrive withoutwaiting for a complete packet from any customer ports. In an embodiment,the invention having packets that are sent using configurable logicblocks (CLB) converted into dedicated logic blocks. In an embodiment,the invention having the CLB being a layer 1+ field-programmable gatearray (FPGA) or application specific integrated circuit (ASIC) of therelay.

In an embodiment, the invention provides relay latency below 1microsecond round-trip, the latency measured through all utilized CLBsand eight network ports including: customer ports port in first network,trunk port out, trunk port in, customer ports port out second networkand then back: customer ports port in second network, trunk port out,trunk port in, customer ports port out first network. In an embodiment,the invention CLB is a layer 1+ field-programmable gate array (FPGA) orapplication specific integrated circuit (ASIC) of a relay device ofclaim 1 for point-to-point wireless, Infiniband or Ethernettransmissions. In an embodiment, the invention provides relay devicelatency that is below 1 microsecond round-trip, the latency measuredthrough all utilized CLBs and eight network ports including: customerports port in first network, trunk port out, trunk port in, customerports port out second network and then back: customer ports port insecond network, trunk port out, trunk port in, customer ports port outfirst network.

In an embodiment, the invention further comprising the step of, at anintended sending or receiving trunk endpoint, a map dividing trunkframes and fragments on the basis of previously allocated time slots,the map comprises a bitmap, wherein each bit corresponds to one of aplurality of timeslots, each bit indicating which customer ports bitfrom which customer ports is assigned to which trunk fragment and whichfragment bit within a trunk frame.

In an embodiment, the invention provides at an intended sending orreceiving trunk endpoint, a map is provided dividing trunk frames andfragments on the basis of previously allocated time slots, the mapcomprises a bitmap, wherein each bit corresponds to one of a pluralityof timeslots, each bit indicating which customer ports bit from whichcustomer ports is assigned to which trunk fragment and which fragmentbit within a trunk frame. In an embodiment, the invention encryptingcustomer ports data that is provided by combining all customer portsbits using XOR scrambler bits, the pro rata share for all customer, themethod used to spread customer ports data within trunk frames and themap which divides up customer ports bits by time slice.

In an embodiment, the invention forwarding any and all customer portspackets to the correct destination is based on the relay port from whichit was received. In an embodiment, the invention forwarding all networkadministration, routing, discovery, query, reply, broadcast, unicast andmulticast packets to the destination port on the second packet-switchingnetwork is provided without changing the configuration or routing of therelay device and without sending those packets anywhere.

Another embodiment of the invention provides for a relay for receivingInternet Protocol (IP), Infiniband or Ethernet packets for multiplecustomers C₁ to C_(n) over a packet-switched computer network. The relaycomprising a First-In First-Out (FIFO) register having sixteen, four bitselectors, creating a 16 bit trunk fragment and filling each trunkfragment with data bits from customer ports C₁ to C_(n) and repeatingthe filling sequence for each 16 bit fragment until sufficient trunkfragments form a filled trunk frame having at least 1,232 customer portsbits plus 16 bits of cyclic redundancy check (CRC) and a frame count anda sender for transmitting each filled trunk frame across a trunknetwork.

In an embodiment, the relay has only idle frames that are sent beforeand after customer port packets to distinguish between customer portsdata and idle frames in order to provide layer 1+ transmission, as allpossible packets of all formats may be sent and are surrounded by idleframes. In an embodiment, the relay provides a filled trunk fragmentthat is 16 bits or smaller and each selector capable of mapping for onlyone customer. In an embodiment, the relay has each customer port C₁ toC_(n) transmit bits received in every filled trunk frame.

In an embodiment, the relay has all customer packets forwarded by thesender to the correct destination based on the relay port from which itwas received.

In an embodiment, the relay has all network administration, routing,discovery, query, reply, broadcast, unicast and multicast packetsforwarded by the sender to the destination port on the secondpacket-switching network without changing the configuration or routingof the relay device and without sending those packets anywhere exceptthe destination port for the sending customer ports. In an embodiment,the relay has each trunk frame fragmented. In an embodiment, the relayhas the trunk network throughput at least 199 Kb.

The invention also provides a low latency logic device comprising aFirst-In First-Out (FIFO) register to capture incoming data packets fromeach customer port in parallel and speeds equal to or greater than 1Gbps, selectors connected to the FIFO outputs, each selector capable ofmapping individual incoming bits from different customer ports andcombining them into the same output trunk fragments in order to createtrunk fragments where the number of bits indexed in each trunk frame foreach customer port is a pro rata share of bandwidth designated to eachcustomer C₁ to C_(n) the selectors filling each trunk fragment with bitsfrom customer ports C₁ to C_(n), by selector values for each customerport, selectors repeating the filling sequence for each fragment untilsufficient trunk fragments form a filled trunk frame, an Exclusive-Or(XOR) Look Up Table (LUT) comprised of 16 to 1232 scrambling bits tocreate scrambled trunk fragments; and a sender for transmitting eachfilled and scrambled trunk frame across a trunk network at atransmission rate that is less than a combination of all customer portincoming bandwidths from a first incoming packet-switched computernetwork.

In an embodiment, the logic device has between 2 and 64 customer portsper logic device and 16 to 64, 4 bit selectors to fill each trunkfragment. In an embodiment, the logic device comprises a firstpacket-switched computer network linked to a trunk network, linked to asecond packet-switched computer network and the filled and scrambledtrunk frame received by the second packet-switched computer network andbeing unscrambled by an XOR LUT. In an embodiment, the logic device hasselectors that receive data bits from every customer port as soon as thefirst bits arrive without waiting for a complete packet from anycustomer port.

In an embodiment, the logic device has packets that are sent usingconfigurable logic blocks (CLB) converted into dedicated logic blocks.In an embodiment, the logic device has the CLB in a layer 1+field-programmable gate array (FPGA) or application specific integratedcircuit (ASIC) of the logic device. In an embodiment, the logic devicehas latency below 1 microsecond round-trip, the latency measured throughall utilized CLBs and eight network ports including: customer ports portin first network, trunk port out, trunk port in, customer ports port outsecond network and then back: customer ports port in second network,trunk port out, trunk port in, customer ports port out first network.

In an embodiment, the logic device has the CLB at a layer 1+field-programmable gate array (FPGA) or application specific integratedcircuit (ASIC) of a logic device device of claim 1 for point-to-pointwireless, Infiniband or Ethernet transmissions. In an embodiment, thelogic device has latency is below 1 microsecond round-trip, the latencymeasured through all utilized CLBs and eight network ports including:customer ports port in first network, trunk port out, trunk port in,customer ports port out second network and then back: customer portsport in second network, trunk port out, trunk port in, customer portsport out first network.

In an embodiment, the logic device comprises the step of, at an intendedsending or receiving trunk endpoint, a map dividing trunk frames andfragments on the basis of previously allocated time slots, the mapcomprises a bitmap, wherein each bit corresponds to one of a pluralityof timeslots, each bit indicating which customer ports bit from whichcustomer ports is assigned to which trunk fragment and which fragmentbit within a trunk frame. In an embodiment, the logic device has anintended sending or receiving trunk endpoint, a map is provided dividingtrunk frames and fragments on the basis of previously allocated timeslots, the map comprises a bitmap, wherein each bit corresponds to oneof a plurality of timeslots, each bit indicating which customer portsbit from which customer ports is assigned to which trunk fragment andwhich fragment bit within a trunk frame. In an embodiment, the logicdevice encrypting customer port data provided by combining all customerbits using XOR scrambler bits, the pro rata share for all customerports, the method used to spread customer ports data within trunk framesand the map which divides up customer ports bits by timeslice.

In an embodiment, the logic device forwarding any and all customerports' packets to the correct destination is based on the logic deviceport from which it was received. In an embodiment, the logic deviceforwarding all network administration, routing, discovery, query, reply,broadcast, unicast and multicast packets to a destination port on thesecond packet-switching network is provided without changing theconfiguration or routing of the logic device and without sending thosepackets anywhere except the destination port for the sending customerport.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 depicts a schematic view of the components present invention;

FIG. 2 depicts a board layout view of components of the presentinvention;

FIG. 3 depicts a schematic view of the functionality of a relay of thepresent invention;

FIG. 4A depicts a flow diagram outlining the operation of an embodimentof the present invention using 4-bit values and a 16 bit frame;

FIG. 4B depicts a flow diagram outlining the operation of an alternateembodiment of the present invention using 6-bit values and a 64 bitFrame;

FIG. 4C depicts a flowing diagram outlining the operation of analternate embodiment of the present invention using 6-bit valves and a64 bit frame and a single selector per customer ports;

FIG. 5A depicts a schematic view of an embodiment of a FIFO registerwith respect to FIG. 4A;

FIG. 5B depicts a schematic view of an embodiment of an alternate FIFOregister of the present invention with respect to FIG. 4B; and

FIG. 6 is a hypothetical calculation of the savings of bits provided bythe present invention.

While the invention is amenable to various modifications and alternateforms, specific embodiments have been shown by way of example in thedrawings and will be described in detail, it should be understood thatthe intention is not to limit the invention to the particularembodiments described. The intention is to cover all modifications,equivalents and alternatives falling within the spirit and the scope ofthe invention.

DETAILED DESCRIPTION

The overall architecture of the present invention may be understood withrespect to FIG. 1. However, other layouts and system architecture andequipment may be used to accomplish the present invention and thecomponents of FIG. 1 provide a single example to accomplish suchinvention. Local Area Network (LAN) A is exemplary of a central routingstation or data center that receives incoming data from multiplecustomer ports C₁-C_(n) via fiber optic or copper cabling 10 (LAN A) or46 (LAN B). In the preferred embodiment, the data is transmitted fromcustomer ports at 10 Gigabit speeds (G) and are received by either aswitch/relay 12 or an appliance or relay that eliminates the switch andcan handle multiple, simultaneous 10 G or faster transmissions. Forexample, many 10 G small form factor pluggable transceivers (SFP) orQuad SFP (QSFP) or SFP+ transceivers may be used by the switch/relay 14,42 or appliance to receive the incoming data. Multiple relays 14 orappliances may be provided at the data center for receiving data fromhundreds or thousands of customers. Each switch is connected to aserver/relay 14, or each appliance. The relay 14 embodies both theswitches and server, preferably also capable of receiving andtransmitting data at 10 G. Each relay 14 or appliance includes multiplenetwork interfaces 16 to send, receive, control and manage the datapackets.

In an embodiment, a logic device such as a Field Programmable Grid Array(FPGA) 16, 50 may provide data management, network interface and packetcontrol. For example, a Xilinx 7 Series FPGA 16 with thousands ofconfigurable logic blocks (CLB) may be programmed to accomplish many ofthe functions of the present invention. (Although “FPGA” is denotedthroughout the application, any functionally equivalent electroniccircuit or logic device may be used for this invention.) Other types ofchips or logic devices may be used for data management, networkinterface and packet control, such as an ASIC. The server/relay 14 orappliance also includes a network interface card (NIC) 18 which aids inthe transmission of the data.

In a preferred embodiment, the data is simulcast via a trunk networkthat may include a radio 20 which transmits using free space optics(FSO) or a microwave (MW) radio frequency (RF) or a millimeter wave(MMW) RF in multiple hops by line of sight transmission of data at 10transmission rate or higher to targeted receivers/antennas viapoint-to-point trunk network 30 b to obtain the data on a second localarea network (LAN) B. In an embodiment, LAN A and LAN B may include apacket switched computer network providing transmissions as 10 Gbs.

In a preferred embodiment, data is simulcast via fiber optic cables 32or a second RF link 30 b using a disparate route to LAN B. A transmitter22 transmits the data across fiber optic cables 32 or second RF link tothe receiving LAN B. A transmitter 22 transmits the data across fiberoptic cables 32 to the receiving LAN B. Due to the high importance ofreceiving data without error and to limited latency, the simulcasting ofdata across wireless transmission 30 b and fiber cables 32 provides theneeded redundancy in order to avoid dropped packets. As is well known,RF transmissions are affected by weather and on bad weather days thefiber optic transmission means 32 (while not as fast as RFtransmissions) may be preferred on such days/stormy periods.

Local area network (LAN) B represents a receiving location, such as astock exchange or electronic trading facility. Wireless signals arereceived by radio 40 and transmitted at 1G or higher speeds via fiberlink 30 c to server/relay 42 or appliance which includes a networkinterface card 48 and a motherboard network interface 48 including a MAClayer, for example, on an FPGA 50. Receiver 52 receives fiber optic orsecond wireless data from the cable 32 which is transmitted to theserver/relay 42. The server/relay is connected to switches 44 orappliances which include transceivers for transmitting data across fiberoptic lines or copper lines 46 to customers. The switches may use QSFPtransceivers, SFP transceivers or SFP+ transceivers to provide 40 Gspeeds or higher. In an embodiment, each customer port C₁-C_(n) has atransceiver for transmitting and receiving data bits/packets.

Turning to FIG. 2, the mother board 60 within the server 14, 42 isdepicted. The board 60 includes a universal serial bus (USB) 62 forreceiving data from a server/relay 90. The data from the USB connectionis transmitted from the board to a single processing unit such as anFPGA 16. The board 16 also includes a Joint Task Action Group (STAG)interface 64 under IEEE 1149.1-1990 provides an interconnect between theUSB and the user logic 66 of the FPGA. The JTAG 64 provides a serialconnection to the FPGA 16 and facilitates programming of the FPGA 16.The functionality of the configurable logic block (CLB) of the userlogic 66 will be explained in more detail below. The media accesscontrol (MAC) 68 receives the instructions from the user logic 66 thatare transmitted via the physical code in sublayer (PCS) then through thephysical medium attachment (PMA). And via the GTX filter circuit 72, 74to transmit the data from the FPGA to the QSFP or SFP+ transceivers 76.

The FPGA 16, 50 may also include an erasable programmable read-onlymemory (EPROM) 78 as a back-up to store the user logic. A memoryinterface MGI and QDR 82 are provided by the FPGA for interfacing withthe user logic 66. Peripheral component interconnect express (PCIE) 84is also provided with the FPGA to interact with the user logic 66.

Turning to FIG. 2, the network interface 16 is preferably a single-chip,32-bit or 64-bit electronic circuit Ethernet controller (e.g. FGPA), andprovides an interface between a local bus of a computer, for example, aperipheral component interconnect (PCIE) 84 local bus, and anEthernet-based gigabit or faster media. The PCIE bus interface unit maybe under the control of the DMA buffer management unit receives datathat is passed to the PCI bus transmit FIFO buffer described below. ThePCIE 84 may be used to dump the bad packets from the receiver 206. Atransceiver 76 may send and receive data packets on the network media atgigabit rates across a physical layer device (e.g., a SFP or fourgigabit serial transceiver).

In alternate embodiments the network interface 16, 50 may also include aPCI bus interface unit, a direct memory access (DMA) buffer managementunit and a network interface portion 16, 50. The network interfaceportion may include an eXtended Gigabit Media Independent Interface(XGMII) 23 b for connecting external 1000 Mb/s or 10000 Mb/stransceivers, an External Address Detection Interface (EADI) 23 c, andan 64b/66b decoder 24. Full-duplex operation can be performed by theXGMII interface. The interface may also include, an LED control and anexpansion bus interface for boot RAM (e.g., EPROM or Flash memory)during startup, and an IEEE 1149.1-compliant JTAG Boundary Scan testaccess port interface. The network interface 16, 50 may also include anetwork port manager and an auto-negotiation unit that communicates viathe media with a corresponding auto-negotiation unit in the hub servingthe network interface with a corresponding auto-negotiation unit in acentralized hub, repeater, or switch that provides shared receivecarrier and collision signals between different network stations.

As depicted in FIG. 2, the GTAG 64 manages the reception of the data bythe network interface unit 16, 50 and retrieves information form headerbytes that are transmitted at the beginning of transmissions. The headerinformation (identifying the byte length of the received frame) ispassed to the FIFO control of the user logic 66.

The above-described MAC 68 may be configured to operate in a shared tengigabit Ethernet network by providing a burst of data packets.Asynchronous data or packetized data can be arranged as datagrams, usingthe User Datagram Protocol (UDP) and the Internet Protocol (IP). UDP/IPare the fragmented datagrams placed in an IP packet format. The UDP/IPpacket is forwarded across a network and the transport and networkinglayer of the OS1 reference model can be sent according to a data layeror physical layer of the OS1 reference model according to the Ethernetprotocol. The datagrams can be removed from the Ethernet protocol andsent using a different protocol if desired. A PHY transceiver and,specifically, the PHY receiver portion, can decode and recognize 64B/66Bencoded data associated with the Ethernet protocol. If the sample rateis 48 MHz, the total bit rate of the network between any two nodes is 48M/frames sec.×64 bytes/frame×8 bits/byte=24.576 Gbits/sec.

When an electronic circuit logic device such as an FPGA 16 is activatedor “powered up,” routing tables are broadcast across the control channelto each of the other devices upon the network. The control channelincludes configuration (or reconfiguration) routing tables needed toaccommodate data transfer between the newly activated device. Therouting table is created to accommodate all of the various channels orframe portions established to receive the various types of data. Data issynchronously sent across the network between activated devices. Therouting table within a memory medium will then identify which byteswithin a frame is associated with a particular channel when subsequentcommunication is desired. In order to use an existing Ethernet PHY, thecompliant network transmission protocol must use 64B/6B coding whichEthernet uses. The recovered clock is available since the Ethernet PHYmust generate it to recover data in Ethernet mode.

The Ethernet MAC 68 may have a learning session Ethernet MAC frame and acompression session Ethernet MAC frame and may be divided into a firstportion comprising two MAC address fields, and a second portioncomprising a rest of the unmodified Ethernet MAC frame.

In an embodiment, the network interface card 18, 48 may have four ports.Port 1 for FR, port 2 for fiber, port 3 for customer ports transmissionsand port 4 for free space optics. However, in an alternate embodiment,the system may be updated to provide between four and thirty-two ports.For example, in a system having 24 ports, there will be 20 extra portsfor customer ports customization. In addition, some of these extra portsmay be used for monitoring and controlling the system using out-of-banddata communications. In addition, the system may have additionalnon-Ethernet ports for communication between multiple instances of thesystem, such as in a Y network layout, where three of the systems willsit in the same rack and communicate between themselves. For example,each of the 20 extra ports may be for a specific customer ports input.In that way the data packets received at each port will have adesignation provided by the port where the data is received. Byproviding port designations to the data, the packet header informationmay be reduced—due to the easier management of data incoming from eachdesignated port. In this way the data packets can be managed at a layerone level requiring less handling at layer two of the Ethernet and lowerlatency.

Turning to FIG. 3, the user logic 66 of the FPGA will be described. Theleft side of the diagram of FIG. 3 represents the electronic circuit orFPGA 16 of the server 14 located in LAN A, including a packet switchedcomputer network and the right side of the diagram of FIG. 3 representsthe electronic circuit or FPGA 50 located in server/relay 42 of LAN Bthat includes a packet switched computer network. The user logic 66 ofthe FPGA 16 includes a distributor 102 for receiving 10 G transmissions100. For example, a 64 bit packet would be received by the distributor102 and is loaded into the First-In-First-Out (FIFO) register 104. TheFIFO 104 functionality will be described in more detail below. Once theFIFO processing is completed, it transfers the 64 bit data to sender 106which transmits the data out of the FPGA as discussed above usingtransceivers such as a QSFP 76 or other transceiver and simulcast asdiscussed above through wireless radio 108, a fiber transmission 110 andfree space optical transmission 112.

These transmissions are monitored by Console 114. In an embodiment, theConsole can also run diagnostic testing at night when the system isinoperable or very few transmissions occur. Configuration block 116 alsointeracts with the user logic and provides for configuration of the FPGA16 when the system is not running—usually at night. A Generation block118 interacts with the Distributor 102 and a clock 120 manages therunning of the FPGA 16. A Compare program 125 also monitors datatransmission between the FPGAs 16 and 50.

The data transmitted from the FPGA 16 from LAN A is transmitted viasimulcast to LAN B (FIG. 1) and is received by the server/relay 42including FPGA 50 (FIG. 3). Wireless data is received by wireless radio208 which transmits the packets to receiver 206, which are loaded intothe FIFO 204. A more detailed description of FIFO 204 will be describedbelow with respect to the present invention. The FIFO 204 transmits itsdata to the Collator 202, which transmits out of the server/relay via 10G speeds, preferably.

The FPGA 50 (FIG. 3) also includes data packet transmissions via fiber210, which is linked to the Receiver 206 for receiving the datatransmission. A free space optical (FSO) transceiver 212 receives fibertransmission and links to the Receiver 206, A Console 214 receivesmonitoring data from the Receiver 206.

The purpose of the packet sharing invention is to lower the latency ofcommunications and to offer a consistent latency regardless of othertraffic on the system, to share the available bandwidth among multiplecustomers using a fixed schedule, to make sure that no network storm ofincoming customer ports traffic will effect the latency of any customerports without reducing the throughput of the system and to make eachpacket transmission more efficient by distributing bits from differentcustomer ports transmissions to fill every wireless bit with data. In atypical round robin type distribution method, some transmission packetsfrom some customers are treated unequally with respect to othercustomers because packets are sent one packet per customer ports at atime. For example, the following service contracts are provided:customer ports one is allotted transmission at 1 megabit per second,customer ports two is allotted 3 megabits per second, customer portsthree is allotted 1 megabit per second, customer ports four is allotted5 megabits per second, and customer ports five is allotted 800 megabitsper second. Due to the standard method of sharing bandwidth none of thecustomers will receive optimal latency and due to the lopsided allotmentof bandwidth to customer ports five, customers 1-4 may have packetsqueued up for considerable time (one-half to several microseconds) undertypical systems, such as round-robin mechanisms. For example, underround-robin scheduling, if each of customers 1-5 have transmissionspackets arriving at the switch 12 (FIG. 1) at the same time, theDistributor 102 (FIG. 3) would queue up every other customer ports whileeach customer ports is serviced, and as a maximum sized packet fromcustomer ports five is being serviced customers one through four wouldwait from 0.5 to 12 microseconds, depending on whether the traffic isfragmented or not. Where IP fragmentation is used, latency of the systemis reduced to 0.5 microseconds, but the throughput of the system is alsoreduced because each IP fragment requires another header. To resolvethis inefficient process, the present invention provides for packetsharing as follows:

The packet sharing invention will be described with respect to FIGS. 4A,B, C and 5A, B. In an embodiment, each trunk fragment may be maximumsized (1500+ bytes) to keep the throughput of the system at a maximum.Each trunk frame every customer ports has some of each customer's bitsin every trunk frame. In other embodiments, each trunk fragment ismaximum sized at 1,248 bits to reduce latency.

The packet sharing invention constructs packets made up of data bytes,data bits and data fragments for each customer port, one bit at a timewith respect to FIG. 5A as follows. While constructing trunk frames(e.g. trunk bits to be transmitted from a first packet switched networkto a trunk network) quickly enough to maintain a completely fullwireless trunk network backbone 30, 32 (FIG. 1). At step 310 a sender105 has a 77×16 bit frame located at sender 106. In an alternateembodiment, a 187×64 bit frame may be located at sender 106 (FIG. 3,5A). At step 320 (FIG. 4A) the sender map uses 16 selectors of the FIFOregister 104 (FIG. 5A) to select which customer ports will fill in eachsender customer data bit. There is selector for each customer port inthe FIFO 104 register (FIG. 3) to fill each trunk fragment sender bit106. At step 330 data bits received from the customer's ports (C1-C16)are transmitted to 16 selectors that take 6 bit values to select eachtrunk fragment sender bit at sender 106. At step 340, for example eachcustomer port transmits bit 4 into the Selector for data fragment senderbit 17. At step 350 the sender has 16 selectors that take 6 bit valuesto select and customer data bits fill each sender 106. For example, atstep 360, you may use the customer ports 7's bit per sender/trunk bit22. The selector combines data bits where the number of data bits ineach trunk frame is a pro rata share of the bandwidth that had beenpreviously designated for each customer port C₁ to C_(n) (where n is thetotal number of customers).

This process repeats at step 365 (FIG. 4A) for each sender 16 bit frameand provides different selector values for each frame. The Selectorcontinues to look for empty bits to fill with each customer portspayload. This repetition of loop 365 starting again at step 320 willcontinue until a custom 1,248 bit trunk frame is ready for apoint-to-point transmission at step 370 from the sender 106 (FIG. 3,5A). Also, selectors can receive data bits from every customer part onthe first incoming packet-switched computer network as soon as the firstdata bits arrive without waiting for a complete data packet from anycustomer part by tracking 4 clock cycles using the standard FIFO (acomplete packet takes at least 16 clock cycles).

Pseudo-code for Packet Sharing Device //// Customers 1 to n all get oneinstance of this. All instances run in parallel. while(incoming_network1_packet )  if( packet_is_not_idle )   RxFifo <=data_packet; // Store data packet in Customer ports's RX FIFO. while(request_for_customer ports_bits )  selector( bits_requested_from_me ) 0: // Do nothing.  1-16: begin   sender <= my_bits; // Could be fromBypass_Data or RxFifo.   my_bits <= my_bits << num_bits; // Shift leftby number of bits retrieved.   if( my_bits < 16 && RxFifo_is_not_empty )   my_bits <= myBits && next_Rx Fifo_entry; // Grab next Fifo entry. End

Data packets are decrypted before being transmitted by the secondnetwork. Each customer fragment must be removed from each packet afterexiting the trunk network. As shown in FIG. 3, the data packets aretransmitted across the trunk network 108, 110, 112, 208, 210, 212 toreceiver 205 to be decrypted by using an XOR LUT at a selector of theFIFO register 204. Collator 202 then transmits the assembled datapackets via transceiver 200, typically at about 10 Gps.

 // In the Packet Sharing Device on the other end of the Trunk  // OneReceiver instance that gets bits from the Trunk and distributes them toall customers.  while( incoming_trunk_bits ) begin    trunk_bits <=Trunk XOR descrambler_mask; // XOR LUT    crc_value <= CRC( trunk_bits,crc_value ); // Keep a running total of what the CRC will be with no biterrors.    if( trunk_bits < 1232 ) begin     Selector( bits_by_customerports )      Cust1 <= trunk_bits;      Cust2 <= trunk_bits;      CustN<= trunk_bits;    end    else begin     if( crc_value != trunk_crc_value)      RecoverDamagedBits();     if( frame_count != trunk_frame_count )begin      bad_frame_count <= bad_frame_count + 1;      good_frame_count<= 0; // Reset the good frame count.      if( bad_frame_count == 255 )begin       Resynchronize();       bad_frame_count <= 0; // Reset thebad frame count.      end     end     else if( bad_frame_count > 0 )begin      good_frame_count <= good_frame_count + 1;      if(good_frame_count > 16 ) begin       bad_frame_count <= 0; // Reset thebad frame count.       good_frame_count <= 0; // Reset the good framecount.      end     end     crc_value <= 0;    // Reset the CRC.    trunk_bits <= 0;    // Reset the Trunk bit count.     frame_count <=frame_count + 1;    // Increment the frame count.   end  // Customers 1to n all get one instance of this. All instances run in parallel. while( incoming_bits_from_trunk ) begin   if( packet_is_not_idle )begin     TxFifo <= incoming_bits_this_customer ports;     if(end_of_data_is_near )      Begin_sending_packet()   end   else    Send_idle();

With respect to FIG. 4A, 5B; an alternate embodiment of the inventionoperates in the following way, while constructing packets quickly enoughto maintain a completely full wireless trunk network backbone 30, 32(FIG. 1). At step 410 (FIG. 4B) a sender 105 has a 187×64 bit framelocated at sender 106 (FIG. 3, 5B). At step 420 (FIG. 5B) the sender mapuses 64 selectors of the FIFO 104 (FIG. 5B) to select which customerports will fill in each sender bit. There is selector for each customerports in the FIFO 104 (FIG. 3) to fill each sender bit 106. At step 430,the data bits received from customer ports (C1-C64) are transmitted to64 selectors that take 6 bit values to select each sender bit at sender106. At step 440 for example, each customer ports puts bit 4 into theselector for sender bit 17. At step 450 the sender has 64 selectors thattake 6 bit values to select and customer ports fills each sender 106.For example, at step 460, the sender uses the customer ports 7's bit persender bit 22. This process repeats at step 465 for each sender 64 bitframe and provides different selector values for each frame. Theselector continues to look for empty bits to fill with each customerports payload. This repetition of loop 465 starting again at step 420will continue until a custom 1,520 byte or larger packet is ready for apoint-to-point transmission at step 470 from the sender 106 (FIG. 3,5B).

In an alternate embodiment, packet sharing will be described withrespect to FIG. 4C. At step 510 the sender 106 (FIG. 3, 5B) creates aframe with 64 possible byte positions. At step 520, a single selectorper customer ports that uses a 6 bit value puts bits in sequence intoproper sender bit locations. At step 530, the sender 106 has 64selectors that take 6 bit values to select and customer ports fills eachsender bit. These steps 520, 530 are repeated for each sender 64 bitframe at step 535 to provide different selector values for each frameuntil a custom 1,520 byte or larger packet is ready for point-to-pointtransmission at step 540 from the sender 106 (FIG. 3, 5). As well, thecustomer ports data may be encrypted by combining all customer portsbits using the XOR scrambler bits, the pro rata share for all customers,the method used to spread customer ports data within trunk frames andthe map which divides up customer ports bits by time slice as shown inthe code below.

 // One sender instance that gets bits from all customers.  while(request_for_trunk_bits ) begin   if( trunk_bits < 1232 ) begin   trunk_bits <= bits_from_cust1 && bits_from_cust2 && ... &&bits_from_custN; // Get bits from each customer ports.    truck_bits <=trunk_bits XOR scrambler_mask; // XOR LUT    Trunk <= trunk_bits;   crc_value <= CRC( trunk_bits, crc_value ); // Keep a running total ofthe CRC.   end   else begin    Trunk <= crc_value && frame_count;   crc_value <= 0;    // Reset the CRC.    trunk_bits <= 0;     // Resetthe Trunk bit count.    frame_count <= frame_count + 1;     //Incrementthe frame count.   end  end

This packet sharing process also provides for encryption in and ofitself. Such a process can be also combined with an encryption map, suchas by Exclusive-Or Look Up Table (XOR'g) the bits with a changingpattern know to both the transmitting and receiving ends but notcommunicated across the wireless link. The encryption step may occur atthe FIFO 104 or sender 106 (FIG. 3, 5C).

The packet sharing systems is also illustrated with respect to FIGS. 5and 6. For example, referring to FIG. 6, each customer ports may have188 entries times 7 bit starting value and 256 entries at 8 bits equal 2Kb. So with 4 customers equals 128 Kb. If the sender has 188 entries for6 bit customer ports at 64 bits each and 188 entries times 384 timesequals 71 Kb for a total of 199 Kb. Such an approach can help to managemultiple customer ports each with different levels of subscribedbandwidth, such that wireless packets can be sent using configurablelogic blocks (CLB) to craft each bit of wireless data all based oncustomer port bandwidth allocation. Under such a method latency isreduced during traffic periods when more than one customer ports istransmitting across the network. Latency can be tested by usingloop-back testing at the PCS 70 and PMA 72 (FIG. 2) to determine roundtrip time on nanoseconds.

The above described functionality may be implemented in anypoint-to-point or serial transmission type system. Numerous variationsand modifications will become apparent to those skilled in the art oncethe above disclosure is fully appreciated. It is intended that theclaims be interpreted to embrace all such variations and modifications.

The invention claimed is:
 1. A method of combining and transmitting datafragments for multiple customer ports C₁ to C_(n) sent over a firstpacket-switched computer network to a trunk network and redistributedover a second packet-switched computer network, each customer porthaving a preselected bandwidth designation, the method comprising thesteps of: providing a First-In First-Out (FIFO) register to captureincoming data bits from each customer port in parallel and speeds equalto or greater than 1 Gbps; providing selectors connected to the FIFOoutputs, each selector capable of mapping individual incoming data bitsfrom different customer ports and combining them into the same trunkfragments where the number of bits in each trunk frame is a pro ratashare of the bandwidth designated to each customer port C₁ to C_(n),that equals a percentage of total subscribed bandwidth divided 1,232bits or greater; filling each trunk fragment with data bits fromcustomer ports C₁ to C_(n), by selector values for each customer port;repeating the filling sequence for each fragment until sufficient trunkfragments form a filled trunk frame; sending each filled trunk frameacross the trunk network having a transmission rate that is less thanthe sum of all customer port incoming bandwidths but equal to the sum ofthe trunk bandwidth allocation for all customer ports; and transmittingthe filled trunk frames from the first incoming packet-switched computernetwork, through the trunk network to the second packet-switchedcomputer network and transmitted to an intended receiving networkendpoint.
 2. The method of claim 1 wherein there are at least 2customers and up to 64 customers connected to the first packet switchedcomputer network.
 3. The method of claim 1, further comprising the stepof filling of 16-bit trunk fragments using 4-bit selectors to selecteach customer ports' data bit.
 4. The method of claim 1, furthercomprising the step of providing selectors that receive data bits fromevery customer port on the first incoming packet-switched computernetwork as soon as the first data bits arrive without waiting for acomplete data packet from any customer port.
 5. The method of claim 1,further comprising the step of scrambling a 16 bit trunk fragment usinga 16 bit Exclusive-Or (XOR) Look Up Table (LUT) comprised of 16 to 1,232scrambling data bits to create a scrambled trunk fragment.
 6. The methodof claim 5, further comprising the step of adding a trunk frame CRC of12-bits to the filled trunk frame, adding a trunk frame 4-bit count tothe filled trunk frame and resetting the selectors, and the XOR LUTbeginning the next trunk frame with no break or idle section betweenfilled trunk frames.
 7. The method of claim 5, further comprising thestep of encrypting customer port data by combining all customer portbits using the XOR scrambler bits according to the pro rata share forall customers, the method used to spread customer port data within trunkframes and the map which divides up customer ports bits by timeslice,sending trunk fragments that are scrambled by the XOR LUT, but notscrambled by a self-organizing scrambling polynomial, and providing areceiver with an XOR LUT to descramble and break up the trunk fragmentsinto data bits for each trunk frame.
 8. The method of claim 1, furthercomprising the step of receiving the trunk fragments at the receivingnetwork end point of the second packet-switched computer network.
 9. Themethod of claim 1, further comprising the step of sending 1,232 databits per 1,248 total bits (equivalent to 154/156 encoding) with nofurther encoding or loss of bandwidth.
 10. The method of claim 1,further comprising the step of providing each customer port with a FIFOregister, each customer port assigned the FIFO register for holdingpartially received customer data packets, providing selectors that matchcustomer port data bits with the corresponding customer port assignedFIFO register, receiving data fragments by the customer port assignedFIFO register and locating an end-of-frame block type field and theend-of-frame is designated as an idle block type field.
 11. The methodof claim 1, further comprising the step of transmitting the filled trunkframe as soon as enough data arrives to allow the continuous sending ofall customer ports' packets, each customer ports' pro rata share ofbandwidth equals the percentage of total subscribed bandwidth divided by1,232 bits and the selector uses 3, 4, 5 or 6 bit indexes to assemble 8bit, 16 bit, 32 bit or 64 bit values to fill each trunk frame.
 12. Themethod of claim 1, wherein the completed packet is between 1,232 bitsand 1,520 bytes.
 13. The method of claim 1, wherein each customer porthas data bits in every trunk frame and the customer ports' bits arespread out within every trunk frame so as to prevent waiting whenmultiple customers send a number of packets all at the same time. 14.The method of claim 1, wherein the trunk network throughput is at least199 Kb.
 15. The method of claim 1, wherein packets are sent usingconfigurable logic blocks (CLB) and wherein the CLB is a layer 1+field-programmable gate array (FPGA) or application specific integratedcircuit (ASIC), the first packet switched computer network providing arelay for point to point wireless, Infiniband or Ethernet transmissions.16. The method of claim 1, wherein latency is below 1 microsecondround-trip, assuming a trunk transmission speed of at least 10 Mbps. 17.The method of claim 1, further comprising the step of, at an intendedreceiving network endpoint, generating a reception map on the basis ofpreviously allocated time slots from other transmitting networkendpoints and the reception map comprises a bitmap, wherein each databit corresponds to one of a plurality of timeslots, each data bitindicating whether that corresponding timeslot has previously beenallocated.
 18. The method of claim 1, further comprising the step ofperiodically synchronizing, as between the transmitting network endpointand the receiving network endpoint, a time period on which a proposedtransmission map is used and without synchronization among networkresources and wherein the synchronizing step comprises the step of usinga connection over which a synchronization signal is transmitted,separate and apart from any network connection and the synchronizingstep comprises the step of transmitting synchronization packets over thenetwork.
 19. A relay for receiving Internet Protocol (IP), Infiniband orEthernet packets for multiple customers C₁ to C_(n) over apacket-switched computer network, comprising: a First-In First-Out(FIFO) register having sixteen, four bit selectors; the selectorsconnected to the FIFO outputs, the selectors providing a fillingsequence by mapping individual incoming bits from different customersports and combining bits into the same output 16 bit trunk fragments inorder to create trunk fragments where the number of bits indexed in eachtrunk frame for each customer port is a pro rata share of bandwidthdesignated to each customer port C₁ to C_(n); that equals a percentageof total subscribed bandwidth divided by 1,232 bits or greater;repeating the filling sequence for each 16 bit trunk fragment untilsufficient trunk fragments form a filled trunk frame having at least1,232 customer port bits plus 16 bits of cyclic redundancy check (CRC)and a frame count; and a sender for transmitting each filled trunk frameacross a trunk network at a transmission rate that is less than acombination of all customer port incoming bandwidths from a firstincoming packet-switched computer network but equal to the sum of thetrunk bandwidth allocation for all customer ports.
 20. The relay ofclaim 19, wherein only idle frames are sent before and after customerport packets to distinguish between customer port data and idle framesin order to provide layer 1+ transmission, as all possible packets ofall formats may be sent and are surrounded by idle frames.
 21. The relayof claim 19, wherein the filled trunk fragment is 16 bits or smaller andeach selector capable of mapping for only one customer.
 22. The relay ofclaim 19, wherein each customer port C₁ to C_(n), transmits bitsreceived in every filled trunk frame.
 23. The relay of claim 19, whereinall customer packets are forwarded by the sender to the correctdestination based on the relay port from which it was received.
 24. Therelay of claim 19, wherein all network administration, routing,discovery, query, reply, broadcast, unicast and multicast packets areforwarded by the sender to the destination port on the secondpacket-switching network without changing the configuration or routingof the relay device and without sending those packets anywhere exceptthe destination port for the sending customer ports.
 25. A low latencylogic device comprising: a First-In First-Out (FIFO) register to captureincoming data packets from each customer port in parallel and speedsequal to or greater than 1 Gbps; selectors connected to the FIFOoutputs, each selector capable of mapping individual incoming bits fromdifferent customer ports and combining the bits into the same outputtrunk fragments in order to create trunk fragments where the number ofbits indexed in each trunk frame for each customer port is a pro ratashare of bandwidth designated to each customer C₁ to C_(n); that equalsa percentage of total subscribed bandwidth divide by 1,232bits orgreater; the selectors filling each trunk fragment with bits fromcustomer ports C₁ to C_(n), by selector values for each customer port;the selectors repeating the filling sequence for each fragment untilsufficient trunk fragments form a filled trunk frame; and a sender fortransmitting each filled trunk frame across a trunk network at atransmission rate that is less than a combination of all customer portincoming bandwidths from a first incoming packet-switched computernetwork but equal to the sum of the trunk bandwidth allocation for allcustomer ports.
 26. The logic device of claim 25, wherein there arebetween 2 and 64 customer ports per logic device and 16 to 64, 4 bitselectors to fill each trunk fragment.
 27. The logic device of claim 25,the first packet-switched computer network linked to the trunk network,linked to a second packet-switched computer network and an Exclusive-Or(XOR) Look Up Table (LUT) compromised of 16 of 1232 scrambling bits tocrate scrambled trunk fragments for filling the trunk frame and thefilled and scrambled trunk frame received by the second packet-switchedcomputer network and being unscrambled by the XOR LUT.
 28. The logicdevice of claim 25, having selectors that receive data bits from everycustomer port as soon as the first bits arrive without waiting for acomplete packet from any customer port.
 29. The logic device of claim25, wherein packets are sent using configurable logic blocks (CLB)converted into dedicated logic blocks and the CLB is a layer 1+field-programmable gate array (FPGA) or application specific integratedcircuit (ASIC) of the logic device.
 30. The logic device of claim 25,wherein the logic device latency is below 1 microsecond round-trip, thelatency measured through all utilized CLBs and eight network portsincluding: customer ports port-in first network, trunk port-out, trunkport-in, customer ports port-out second network and then back, customerports port-in second network, trunk port-out, trunk port-in, customerports port-out first network and the CLB is a layer 1+field-programmable gate array (FPGA) or application specific integratedcircuit (ASIC) of a logic device device of claim 1 for point-to-pointwireless, Infiniband or Ethernet transmissions.